Oct 03, 2013
The robots.txt file is used to instruct search engine robots about what pages on your website should be crawled and consequently indexed. Most websites have..